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Abstract — Low-density parity-check (LDPC) coding for a mul- 
titude of equal-capacity channels is studied. First, based on 
numerous observations, a conjecture is stated that when the 
belief propagation decoder converges on a set of equal-capacity 
channels, it would also converge on any convex combination of 
those channels. Then, it is proved that when the stability condition 
is satisfied for a number of channels, it is also satisfied for any 
channel in their convex hull. For the purpose of code design, 
a method is proposed which can decompose every symmetric 
channel with capacity C into a set of identical-capacity basis 
channels. We expect codes that work on the basis channels to be 
suitable for any channel with capacity C. Such codes are found 
and in comparison with existing LDPC codes that are designed 
for specific channels, our codes obtain considerable coding gains 
when used across a multitude of channels. 

I. Introduction 

Design of codes for specific channels is a mature subject. 
A code optimized for one channel, however, may not perform 
well if used on other channel types [1]. Thus, recently, there 
has been an increasing interest in universal codes, i.e., codes 
that perform well on a multitude of channels [2]-[4]. Such 
codes reduce system complexity by removing a need for 
frequent code changes in the system and by allowing for once- 
and-for-all coding solutions. 

Low-density parity-check (LDPC) codes [5] are extremely 
strong error correcting codes. Interestingly, various authors 
have observed "universal properties" of these codes [3], [4], 
[6]— [8]. Chung [6] points out that LDPC codes optimized for 
the Gaussian channel perform well on some other channels 
such as the Rayleigh channel. In a more general setup, Shi 
and Wesel [4] discuss the universal properties of finite block 
length codes. 

Peng et al. [8] design LDPC codes for a number of channels 
(in this case, the Gaussian channel, the binary erasure channel 
(BEC) and the Rayleigh channel). They argue that for a set of 
channels, usually one channel can be taken as the surrogate. 
They design the code only for the surrogate channel. This 
code works satisfactorily on all the given channels, but not 
necessarily on other channel types. 

Despite some universal properties of LDPC codes, the 
performance of a code designed for one channel, can be poor 
on another channel with the same capacity. For example, a 
rate one-half LDPC code with maximum node degree of 100 
(taken from [9]), achieving more than 99.7% of the capacity of 
a BEC with capacity C = 0.5, does not converge on a binary 
symmetric channel (BSC) with capacity C = 0.63. In other 



words, this code does not achieve even 80% of the capacity 
of the BSC. 

In this work, we design LDPC codes that have strong 
universal properties. For a given channel capacity C, we find 
LDPC codes that perform well on any channel with this 
capacity. The main body of this paper consists of: (1) studying 
code design and stability analysis for convex combination of 
N channels of equal capacity, and (2) decomposition of all 
channels with capacity C into a number of basis channels of 
the same capacity. 

In Section Hill we study code design for the convex com- 
binations of a set of equal-capacity channels (see Section ITlTl 
for definition). We conjecture that under belief propagation 
decoding, to design a code for all the convex combinations 
of N binary-input symmetric-output (BISO) channels, it is 
sufficient to find a code which converges only on those N 
channels. We also prove that when stability condition [10] is 
satisfied for N BISO channels, it is also satisfied for all their 
convex combinations. As a result, if a set of basis channels 
with capacity C can be found, a code designed only for the 
basis channels has strong universal properties. 

In Section|lV] a channel decomposition method is suggested 
which decomposes any BISO channel with capacity C in 
terms of basis channels of the same capacity with nonnegative 
coefficients. This decomposition method is not similar to 
existing techniques that decompose the channel over BSCs of 
various capacities [11]. Our technique is exhaustive, i.e., any 
channel with a given capacity can be spanned with nonnegative 
coefficients over our suggested basis. 

For any given capacity C, a code designed only for our 
suggested basis channels is expected to have strong universal 
behavior over all channels with capacity C. Examples are 
provided in Section [V] Simulations confirm that compared to 
codes designed for specific channels, our codes have signifi- 
cantly better universal properties. 

II. Preliminaries: LDPC Code Design and 
Symmetric Channel Representations 

In this paper, an ensemble of LDPC codes is defined by a 
pair of distributions (A, p) in the polynomial form, i.e., \(x) = 

Si>2^ a;I ~ 1 and P( x ) = Yti^Pi^ 1 ' where A; (pi) is the 
fraction of edges connected to the variable (check) nodes of 
degree i. 

LDPC code design means optimizing degree distributions 
in order to maximize a cost function which can be usually 
the code threshold or the code rate. In threshold maximization 



given a fixed rate, we seek a code working under the worst 
channel condition, hence minimum gap between the given 
rate and channel capacity. In rate maximization given a fixed 
channel, the code which exhibits maximum rate and provides 
reliable communication over the given channel is found. 

In this work, we focus on finding codes suitable for a 
multitude of channels rather than a single channel. We adopt 
the rate maximization approach which allows us to find a 
code working on a finite number of channels. Using the rate 
maximization approach, code design for a number of identical- 
capacity channels is similar to what has been discussed in [12]. 

We assume that transmission of LDPC codes takes place on 
a memoryless BISO channel, i.e., (X, W(Y\X),y) where the 
input alphabet X = {0, 1} is uniformly distributed, W(F|X) 
is the conditional pdf of the channel and J C 1 is the 
output alphabet of the channel. In this section, three channel 
representations for a BISO channel are presented. A BISO 
channel is a channel for which we have 

W{Y\X = 1) = W{-Y\X = 0). 

At the receiver side, given the output Y, the sufficient 
statistics for X is Pr(X = Q\Y) which depends only on the 
conditional pdf of the channel. For a symmetric channel, the 
log-likelihood ratio (LLR) is defined as 



M = log 



PvjX = 0\Y) 



Prpf = 1\Y) 



It is straightforward to see that M is another sufficient statistics 
for X, i.e., I(X;Y) = I(X;M). Assuming that the all- 
zero codeword is transmitted, we denote the pdf of M as 
a(x) which fully characterizes the channel and satisfies the 
symmetric property a(— x) = e~ x a(x) [10]. Since the channel 
is symmetric and input symbols are equiprobable, the mutual 
information is the same as capacity [13]. We denote the mutual 
information I(X\ Y) associated with a(a;) by X(aJ] defined as 



1(a) =1-1 a{x) log 2 (l + e~ x )dx. 



Let 



P = min |Pr(X = 0\Y),~Pr(X = 1\Y)\ 



be a random variable over [0, \] with the pdf denoted by g(p) 
which completely characterizes the channel [11]. This, in fact, 
is the probability of error based on observation of the output. 
Since the pdf of P, i.e., g(p), should not satisfy any symmetry 
property, we will use this representation in Section [TV] 

Let A z (x) be the Dirac delta function at x = z. Also, for an 
e G [0, 1], define e = 1 — e. A BSC with crossover probability 
e can be represented by the pdf of LLR and the pdf of error 
probability as 

a(x) = eA_ log( |)(V) + eA log( |)(x) 

and g(p) — A e (p), respectively. 

1 For ease of notations, sometimes the argument of LLR densities is dropped. 
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Fig. 1 . Convex combination of two subchannels where each bit is transmitted 
through CHi with the probability of 7. The LLR densities of subchannels are 
Po and %• 



Every valid pdf over p G [0, 5] can be seen as a symmetric 
channel. In fact, this representation decomposes a symmetric 
channel into the convex hull of BSCs of different crossover 
probabilities. The fraction of bits passing through each BSC 
is captured in g(p). According to [11], the channel capacity is 



e = E{e BS c (P )} = i 



h(p)g(p)dp 



where h(p) is the binary entropy function and integration 
includes the mass points at p = and ~. Also, g(p) is obtained 
by 



1 



p(i -p) 



a,,,, [log^— ^ ) , VpG [O,^] 



where a| m |(x) is the pdf of the absolute value of LLR, i.e., 

a M (.x) = a(x)(l + e~ x ), Vx > 0. 

The three channel representations presented in this section 
are interchangeable as long as the input is uniformly dis- 
tributed [11]. 

III. Universal Codes for Convex Combination of 
Two Channels and Stability Analysis 

In this section, we consider universal codes for convex 
combination of two channels. We start with the definition of 
convex combination of two channels. 

Fig. Q] illustrates a channel composed of two subchannels 
of capacity C and LLR densities p and q . Each bit is passed 
through subchannel 1 with probability 7 G [0,1] or passed 
through the other subchannel with probability of 1 — 7. We 
refer to this channel as a convex combination of subchannels 1 
and 2 since the LLR density for this combination is 7p +7q . 
The same equation holds for the pdf of error probability. We 
denote the set of channels in the convex hull of p and q by 

£(p ;qo) = {7P + 7q l7e [0,1]}. 

Since both of the subchannels have capacity C, for every 
channel a 7 G £(p ,q ), /(a 7 ) = C. 

The setup in Fig. Q] can be viewed as an arbitrarily varying 
channel [14] with a Bernoulli distributed set of states. As a 
straightforward generalization, we can extend this configura- 
tion to N subchannels. 



Throughout this paper, we assume that the receiver has the 
channel state information and computes correct LLRs. If 7 is 
fixed, one can design a single good LDPC code for the channel 
with LLR density of 7p +7q - However, if 7 varies over time, 
finding a code which works for all 7 G [0, 1] is a challenging 
problem. In fact, we seek a single universal code which does 
not need to be changed over time, even with variations of 7. 

Remark 1: If the transmitter knew through which of the 
subchannels the next transmitted bit would pass (which is not 
the case in most practical situations), it could use two different 
codes for two subchannels. 

A. Code Design for Two Channels 

For code design, we use density evolution [10]. Consider a 
degree distribution pair (A, p) and a BISO channel with LLR 
density ao which can be either p or q . Density evolution is 
stated as 

&t = a ® Aj ^2 Pk&t-x ] (1) 

i ^ k ' 

where and (•) K| ( fe - 1 ) denote the operation of a 

degree-z variable node and a degree-fc check node, respec- 
tively. 

Let us first assume that there are only two possible cases of 
7 = and 7=1. Similar to code design for one channel, and 
by considering the convergence constraints on both channels, 
codes that are suitable for both channels can be designed. 
Similarly, if 7 takes only a finite number of values, the same 
approach can be used. While the objective function remains 
the same, the number of constraints grows almost linearly with 
the number of channels. Thus, this approach can be inefficient 
when 7 can take many values. In general 7 may take all values 
in [0,1]. 

Our numerous experiments show that if we guarantee con- 
vergence for two values of 7, the code will converge for all the 
values of 7 in between. Therefore, in order to design a code for 
the channel model given in Fig. Q] when 7 is unknown, we 
force convergence of the code only for 7 = and 7 = 1. 
Our extensive experimental results have been consistent in 
verifying the following conjecture. 

Conjecture 1: An LDPC code which converges over two 
channels with LLR pdfs p and q , will also converge over 
any channel in the convex hull of those channels, i.e., over all 
a 7 G £(p ,q )- 

Remark 2: The above conjecture can be proved under the 
assumption that the distribution of the output messages of 
variable nodes depends only on the information they bear 
and the structure of the code. According to central limit 
theorem, the distribution of random messages at the output 
of a variable node is close to a Gaussian distribution. Thus, 
the input distribution of a check node is a mixture of Gaussian 
distributions [6], [15] which is uniquely characterizes by the 
degree distributions and the input mutual information. It is 
noticeable that in this approach, we do not assume that the 



output distributions of the check nodes are Gaussian, which 
is a usual assumption in the literature [6]. Therefore, we 
expect codes designed based on this conjecture to have strong 
universal properties. 

B. Stability Analysis 

Convergence analysis of an LDPC code over a symmetric 
channel is often performed by characterizing the fixed points 
of density evolution given in ([TJ. Let 

5(a) -/«x)e^* 

and 

P(a) = i / a^e-d^l+^W 

be the Bhattacharyya constant and error probability associated 
with a symmetric density a(x), respectively. It is noticeable 
that these two functionals are linear with respect to symmetric 
densities. Also, it is well-known that 

27>(a) < 0(a) < 2 [P(a)P(a)] = . (2) 

From (Q}, one can see that for every symmetric channel, Aoo is 
a fixed point of density evolution corresponding to the perfect 
decoding. It is desirable that the perfect decoding fixed point 
be stable, which means that once the probability of error gets 
small enough, the decoder converges to the error-free density 
(Aqo) [13]. The following theorem shows the stability of A^. 

Theorem 1: For every channel in £(p ,q ), once the de- 
coder gets close to the perfect decoding, it will converge to 
the error-free fixed point (Aqo), provided that the given LDPC 
code converges on two subchannels p and q . 

Proof: Let a = [7P + 7q ] G £(p , q ) for a 7 e [0, 1]. 
According to [13], the Bhattacharyya functional is multiplica- 
tive under © operation. Also, it can be shown that for two 
symmetric densities u and v, we have 

B(u H v) < B(u) + B(v) - B(u)B(v) 

where the upper bound becomes tight for two BEC channels. 
Hence, by induction it is straightforward to see that 

B(u m ) < 1 - (1 - B(u)) fc . 

Letting ti = B{&e), we arrive at 

h < B(ao)A(l - p(l - t<_i)) (3) 

which coincides with the density evolution for the BEC by 
substituting the inequality with an equality. Expanding OJ 
around zero, we get 

U < B(ao)A'(oy(l)fc_ x + 0{t}_ 1 ). 

From the necessity part of the stability condition theorem 
for the belief propagation decoding [10], [16], since the given 
LDPC code converges over the two subchannels, we must have 

B(p o )A'(0)p'(l) < 1 



and 

S(q )A'(Oy(l) < 1, 

hence £>(ao)A'(0)//(l) < 1. One can see that there exist 
fii > and ^ 2 > such that B(p o )A'(0)p'(l) + m < 1 
and B(q )A'(0)p'(l)+/i 2 < 1. Therefore, 

B(a )A'(oy(l)+M<l 

where [i = 7^1 +7M2 an d /i > 0. For sufficiently small r > 
and some ff, t^_i < r, i.e., getting close enough to the 
perfect decoding, results in 

U < (S(a )A'(0)p'(l) + n)t t -i < U-i. 

It shows that as £ — > 00, ^ — > 0. For a small error 
probability, from (O we have 



where 



/ ■ 

JV(C) 



<p(x, y)dxdy = 1. 



2P(a) < B(a) < 2^P(a)- 

Now let ( = r 2 /4. If V(u) < C» then < r - Thus > as f- tends 
to infinity, 7 , (a£) — > and ae — > Aoo. ■ 

The result can immediately be extended to a multitude of 
subchannels and their convex hull. 

IV. Channel Decomposition 

In this section, we propose a channel decomposition method 
based on the pdf of error probability defined in Section [II] We 
show that every symmetric channel with capacity C can be 
decomposed into a number of basis channels with the same 
capacity. By the result of Section Hill a code that works on the 
basis channels with capacity C is expected to be suitable for 
all channels in the convex hull of those basis channels. 

We seek an identical-capacity basis for a given capacity C 
and contend that the sought basis is in the form of 

9x, y (p) = a(x,y)A x (p) + a(x,y)A y (p) 

where a(x, y) £ [0, 1] is a constant depending on the channel 
capacity and the location of mass points x and y. Since the 
basis channel g x ,y(p) must have capacity of C, by defining 
H = 1 — C we have a(x,y)h(x) + a(x,y)h(y) — H which 
results in 

a(x,y)= hiy) ~ n ■ 
h(y) - h(x) 

Let £ = h^ 1 (fi) be the crossover probability of a single 
BSC with capacity C where function h^ 1 : [0, 1] ^ [0, i] is 
the inverse of the binary entropy function. By the capacity 
constraint, in order to have a nonnegative a(x,y), we must 
have mm{x,y} < £ and max{x,y} > £. Without loss of 
generality we assume that x < y. 

Now we prove that every symmetric channel with capacity 
C falls into the convex hull of the basis channels 

g(C) = {g x>v (p)\(x,y)eV(C)} 

where V(C) = {(x, y) e [0, £] x [f , To this end, we find a 
two dimensional pdf tp(x,y) which fully describes the channel 
according to 



(4) 



The following theorem shows that for any symmetric chan- 
nel, there exists a two dimensional pdf (f(x, y). Before stating 
the theorem, it is important to note that if the given channel 
has a mass point at p = £, then for some 6 <G [0, 1], we can 
write its pdf as g(p) — SA^(p) + 5r(p) and proceed with r(p) 
which does not have any mass point at p = £. 

Theorem 2: Given a capacity C, every pdf of error proba- 
bility g(p) associated with a symmetric channel can be written 

as 



g(p)= ip{x,y)g XtV (p)dxdy, 
Jv(c) 

for some pdf ip(x,y) defined over V(C) — {(x,y) G [0, £] x 
where £ = h~ l (W). 



Proof: Defining 

<p(x,y) = 



ge(x)g r (y) 



(p) = / t f(x,y)g x ,y(p)dxdy 

JV(C) 



one can verify that (p(x, y) is a valid pdf and (01 holds. More 
discussions on ip(x, y) and detailed proof are provided in [17]. 



Without loss of generality and for practical reasons, we 
assume that \y\ is finite. Having a finite alphabet also means 
having a finite basis, because the number of channels that sat- 
isfy is finite. Assuming that the quantization levels are such 
that there are Nt levels less than £ and N r levels greater than 
£, the cardinality of basis channels set is |<?(C)| = Ne x N r . 

V. Simulation Results 

Given the channel capacity C and the number of quantiza- 
tion levels, we can determine the set of basis channels 0(C) 
for which we can design a single LDPC code. According to 
Conjecture [T] this code is expected to exhibit good perfor- 
mance on all channels in the convex hull of basis channels, i.e., 
£[{?(C)] ■ For a given code, we define the universal threshold 
C* as the minimum capacity on which convergence of the 
code is guaranteed. Using Conjecture [TJ and Theorem [2] the 
universal threshold C* can be found by a binary search. 

For all the simulations in this section, we have used 6- 
bit resolution to quantize subchannels (i.e., \y\ = 64 at the 
receiver) and a 9-bit sum-product decoder [18]. We follow the 
guidelines in the literature and consider only two consecutive 
degrees for p(x). Better universal codes can be obtained 
by optimizing p(x), but the performance loss due to this 
simplifying assumption is minor. 

Fig. |2]depicts a bit error rate comparison between a rate 0.6 
code optimized for an additive white Gaussian noise (AWGN) 
channel with a maximum variable node degree of 100 [9], and 
a rate 0.6 universal code with the suggested method having 
maximum variable node of 50. Despite the fact that the AWGN 
code performs slightly better on the AWGN channel, we can 
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Fig. 2. Comparison between bit error rate of a rate 0.6 universal code and 
a rate 0.6 code designed specifically for AWGN on various channels. Both 
codes are randomly constructed and 100, 000 bits long. 



see that the universal code outperforms the AWGN code on 
other channels. 

The universal threshold for the AWGN code is C* = 0.662 
while the universal threshold of the less complex universal 
code is C* = 0.628. Using density evolution, we tested 
both codes on 10,000 randomly generated channels with the 
capacity of their universal threshold. The codes performed well 
on all channels. 

It can be seen that the universal code has similar perfor- 
mance on all channels whereas the code that is designed for a 
specific channel only performs well on the channel for which 
it is designed. Degree distributions for the AWGN code taken 
from [9] are 

p{x) = 0.5a; 12 + 0.5a; 13 , 

X(x) = 0.1499a; + 0.1621a; 2 + 0.0224a; 5 + 0.1764a; 6 



+ 0.0077a; 7 + 0.1166a; 16 + 0.0307a; 2 



0.0438a; 30 + 0.02 78a 



while for the universal code are 



- 0.0048; 



- 0.0319a; 28 
f 0.2258a; 99 



12 



p[x) = 0.5806a; 11 + 0.4194a; 
A(x) = 0.1689a; + 0.1924a; 2 + 0.0604a; 5 + 0.2069a; 6 
+ 0.0763a; 10 + 0.0457a; 29 + 0.24 9 5a; 49 . 

VI. Conclusion 

Design of universal LDPC codes over symmetric channels 
was discussed. We conjectured that, codes that converge on a 
set of equal-capacity channels, also converge on the convex 
hull of those channels. Therefore, we expect codes that are 
designed using this result to exhibit stronger universal proper- 
ties than codes designed for specific channels. We also proved 
the stability of decoder in the convex hull of a set of equal- 
capacity channels. In other words, for a code which converges 
on a number of identical-capacity channels and on any channel 



in the convex hull of these channels, if the decoder gets close 
to the perfect decoding, then it will converge. 

We proposed a channel decomposition technique which 
allowed for spanning any given channel with capacity C on 
a number of basis channels with identical capacity. Then, 
we designed codes for those basis channels. As expected, 
the codes designed following this method exhibited strong 
universal performance. Specifically, in comparison with ex- 
isting LDPC codes designed for a given channel, significant 
performance gain was obtained when transmission took place 
over various channels of equal capacity. Defining a universal 
threshold for a code, we observed that our codes have better 
universal threshold compared to codes designed for specific 
channels. 
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